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^D Abstract 

f^ Motivated by a model for the perception of textures by the visual cortex in primates, we analyse the bifurca- 

^SJ tion of periodic patterns for nonlinear equations describing the state of a system defined on the space of structure 

I tensors, when these equations are further invariant with respect to the isometrics of this space. We show that 

^ the problem reduces to a bifurcation problem in the hyperbolic plane T) (Poincare disc). We make use of the 

^~5 concept of periodic lattice in V to further reduce the problem to one on a compact Riemann surface V /V, where 

t/~) r is a cocompact, torsion-free Fuchsian group. The knowledge of the symmetry group of this surface allows to 

T-H carry out the machinery of equivariant bifurcation theory. Solutions which generically bifurcate are called "H- 

planforms", by analogy with the "planforms" introduced for pattern formation in Euclidean space. This concept 

is applied to the case of an octagonal periodic pattern, where we are able to classify all possible H-planforms 

satisfying the hypotheses of the Equivariant Branching Lemma. These patterns are however not straightforward 

to compute, even numerically, and in the last section we describe a method for computation illustrated with a 

(-H selection of images of octagonal H-planforms. 

C^ Keywords: Equivariant bifurcation analysis; neural fields; Poincare disc; periodic lattices; hyperbolic planforms; 

H irreductible representations; Laplace-Beltrami operator 

^ 1 Introduction 

On 

T^lj- In a recent paper [ . ] a model for the visual perception of textures by the cortex was proposed, which assumes that 

CN populations of neurons in each so-called "hypercolumn" of the visual cortex layer VI are sensitive to informations 

r — . carried by the structure tensor of the image (in fact, of a small part of it since each hypercolumn is dedicated to a 

f^ specific part of the visual field). The structure tensor is a 2 x 2 positive definite, symmetric matrix, the eigenvalues 

^^ of which characterize to some extent the image geometric properties like the presence and position of an edge, the 

^^ contrast, etc. This led to write the average membrane potential V of the populations of neurons in a hypercolumn 

J> as a function of the structure tensor (and time). The equations that govern the evolution of the average membrane 
potential are 

dV f 

^ -Q^{T,T)^~aV{J,T)+ j w{T,T')S{V{r',T))dr +I{T,t) (1) 

where T, T' G SDP{2) (the space of structure tensors), S* is a smooth function M — > M of sigmoid type 
{S{x) — > ±1 as X — > ±oo), / corresponds to some input signal coming from different brain areas such as the 
thalamus and w{T, T') is a function which expresses the interaction between the populations of neurons of types 
T and T' in the hypercolumn. We can introduce a distance on the set of structure tensors by noting that it has the 
structure of a homogeneous space for the group of invertible matrices GL(2, R) acting by coordinate change on 
quadratic forms associated with the structure tensors [ ' ']: for any T € SDP{2) and G g GL(2,IR), we define 
G T — ^GT G. This induces a Riemannian metric on the tangent space: gri^, B) = tr(T~^AT~^i?) and the 
corresponding distance in SDP{2) is given by (see [.32]) 



X 



do (Ti , Ta) = V log2 Ai + log" A2 (2) 



where Ai, A2 are the eigenvalues of 7^~^72- Because of its invariance this distance is biologically plausible 
since the neurons have no obvious way of knowing in which coordinate system are expressed the components of 
the structure tensors and this leads to the assumption that the function w is insensitive to coordinate changes in 
SDP{2). This implies that w is in fact a function of the distance do- 

wiT,r)^fidoiT,T')) (3) 

As a consequence, the equation (1) is invariant under the action of the isometry group GL(2, M). 

Assume now that there is no input: / = 0, and that V^ = is a solution of (I) (we can assume w.l.o.g. that 
S{0) — 0). If the slope /i = ^"(O) is small enough, small perturbations of this basic state are damped to and the 
basic state is stable. We can however expect that if /i exceeds a critical value an instability will grow and lead to 
a new state which need not be invariant under the action of GL(2, M). In other word we expect pattern formation 
through spontaneous symmetry breaking, a phenomenon which has been widely studied in other contexts in the 
last 40 years (see [ ] for a review). From the mathematical point of view the theoretical framework is the 
well-established bifurcation theory with symmetry [21], [12]. There is however a basic difference between the 
problem at hand and previous studies in pattern formation and symmetry-breaking bifurcation: the symmetries 
(isometrics) are not Euclidean. Moreover the group GL(2, M) being non compact, the spectrum of the equivariant 
linear operators is continuous and eigenvalues have infinite multiplicity in general. This is also true when the 
symmetry group is the group of displacements and reflections in Euclidean space. In this case a rational approach 
is to resctrict the analysis to the bifurcation of patterns which are periodic (in space) with a period equal to 27r/fc 
where k is the wave number of the most unstable Fourier modes. This hypothesis is supported by observations 
(to some extent) and it provides a framework which is suitable for the bifurcation analysis. Namely, the spectrum 
of operators restricted to functions which are invariant on a periodic lattice is discrete and consists of eigenvalues 
with finite multiplicity. This allows to fully exploit the power of equivariant bifurcation theory and to describe the 
different types of bifurcated solutions by their (remaining) symmetries. 

We would like to proceed in a similar way for equation (1), that is to consider solutions which are periodic 
in the space of structure tensors (one has to give a precise meaning to this statement) and then apply equivariant 
bifurcation methods to describe the set of solutions. We are not addressing here the question of the relevance of 
looking for such solutions, neither from the neurophysiological point of view, nor even from the point of view of 
their stability under perturbations which do not have the same periodicity (or no periodicity at all). Let us mention 
the fact that in [ ] we identified families of subgroups of the group of isometrics of the set of structure tensors 
that naturally arose from the analysis of the retinal input to the hypercolumns in visual area VI. These subgroups, 
which we called neuronal Fuchsian groups, have the property that the membrane potential functions are invariant 
under their action. This is one example of the neurophysiological relevance of looking for solutions of ( 1 ) that are 
periodic in this sense but it certainly does not give the final answer: These difficult questions will be addressed 
subsequently. Let us point out that, on the other hand, a classification of possible periodic patterns should be 
largely independent of the model equations as long as these equations share some basic properties (like GL(2, ]R) 
invariance). In this paper we shall therefore not focus on equation (1) and our results would apply equally to other 
models such as reaction-diffusion or Swift-Hohenberg equations with Laplace-Beltrami operator in SDP{2)). 

The structure of the paper is as follows: 

1. In section 2 we introduce the necessary basic material from hyperbolic geometry. Periodic lattices and 
functions are defined in SDP{2) and it will be shown that the problem can be decoupled and reduced to 
looking for periodic patterns in the hyperbolic plane (we shall work with its Poincare disc representation 
V). 

2. In section 3 we set the bifurcation problem for periodic patterns in V. Applying the equivariant branch- 
ing lemma [ ] (or isolated stratum principle in the variational case), this leads us to defining hyperbolic 
planforms, which are the hyperbolic counterparts of the planforms defined in the Euclidean case (see [ ]). 

3. In section 4 the methods of section 3 are applied to the case of a regular octagonal pattern for which we can 
describe all planforms which result from the applicaiton of the equivariant branching lemma. 

4. In section 5 we compute eigenvalues and eigenfunctions of the Laplace-Beltrami operator in the hyperbolic 
octagon in order to exhibit hyperbolic planforms which satisfy certain isotropy conditions. 



2 Periodic lattices and functions in the space of structure tensors 
2.1 The space SDP(2) 

The following decomposition will prove to be very convenient. Let ^^ be the determinant of T G SDP{2). We 
set ^ > by convention. Writing T = ^V , we see that SDP{2) == M+ x SSDP{2) where SSDP{2) denotes 
the subspace of tensors with determinant 1. In the open cone SDP{2) the surface SSDP{2) is a hyperboloid 
sheet and it can be shown that it carries a metric induced by the metric g which is just the usual metric of the 
hyperbolic plane. The isometry group of displacements in SSDP{2) is the special linear group S'L(2,M), and 
indeed we may also write the group of orientation preserving isometrics GL+(2, M) = SL(2, M) x M+ acting on 
a tensor T — (T', £,) by 

(4) 



(r,a).(r',e) = (*rr'r,a20- 



We now identify SDP{2) with the "half open cylinder T) x 
following change of variables. 
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where V is the Poincare disc, through the 
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with xi > and xia::2 



Xg > 0. We set 



Xi = zgij, i = 1, 2, 3, Z3 > 0, £1^2 -^3 = 1- 
Now the hyperbolic plane SSDP{2) is further identified with T) through the change of coordinates 

Xi - 

X2 -■ 
X3 ' 

These formulas define a diffeomorphism 

e : (xi,X2,X3) e SDP{2) ^ izi,Z2,Z3) eVx 
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The group of direct isometrics (displacements) in 2? x 

1^,1 2 



+ is SU{1, 1) X M+ where SU{1, 1) = 
n-ii — i/^i^ = 1. Including orientation reversing isometrics would replace SU{1, 1) by C/(l, 1). 
Proposition 1. The metric carried to V x M+ by Q is (up to a normalization coefficient) 
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Proof. The fact that the decomposition is diagonal follows directly from the direct product decomposition and 
group action (4). In order to compute the precise expression for the metric, let us first express the form g in 
SDP{2) in suitable coordinates of the tangent space (which is the space of symmetric 2x2 matrices). Such a 
basis is given, at any tensor T, by 
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Then a straightforward calculation shows that 

g{xi,X2,X3) — z^ [x2dxi + 2x2dxidx2 — 4:X2X3dxidx3 

+x\dx\ — 4:XiX3dx2dx3 + 2(xia;2 + a;3)dx3] 

The proposition follows by applying the pull-back Q* to this form. 
Corollary 1. The volume element in the Zj coordinates is 

4 dzi dz2 dz3 
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Corollary 2. The Laplace -Beltrami operator in T) x IR+ in Zj coordinates is 
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4 V^-^:^ ' 9z| 

We note Ajj the first term on the r.h.s. 

Let us now compute the distance dmT) x IR+. We set z = (zi, Z2, 2:3) and z' = [z'^^ z'^-, z'^). Then 
Proposition 2. 

d(z, Z') = y(d2((^l,^2),(4,4))'+l0g'(J) (10) 

where di denotes the distance in the Poincare disc. 

Proof. We start from expression (2) which gives the distance between two tensors 7i and 72- Let T — z^T, 
V = z'^f' and a = zs/z'^. We note A, 1/A the eigenvalues of f-^f'. Then 

doiT,T'f = log^aA + log2 ^ = 21og2 a + 21og2 A = 21og2 a + do{f,f'f. 

A 

But do{T,T') = d2{{zi, Z2), {z'l, Z2)), hence the result. D 

Note that the distance in D is given by the following formula where z, z' are complex numbers in the unit disc. 

|z-z'| 



d-i^z^z') — 2arctanh 



|l-zz'|' 



(11) 



2.2 Periodic lattices in P x IR+ 

Let us recall first some basic properties of the group of isometrics in the hyperboUc plane (which we shall always 
identify with the Poincare disc T) in the sequel). We refer to textbooks in hyperbolic geometry for details and 
proofs. The direct isometrics (preserving the orientation) in T) are the elements of the special unitary group, noted 
SU(1, 1), of 2 X 2 Hermitian matrices with determinant equal to 1. Given 
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such that lap- |<3P = 1, 



the corresponding isometry 7 is defined by 



7 • z : 



az + /3 
pz + a 



z ev 



(12) 



Orientation reversing isometrics of V are obtained by composing any transformation (12) with the reflection 
K : z !—>■ z. The full symmetry group of the Poincare disc is therefore 

U(l,l) = SU(1,1)Uk-SU(1,1). 

These isometrics preserve angles, however they do not transform straight Unes into straight lines. Given two 
points z 7^ z' in V, there is a unique geodesic passing through them: the portion in V of the circle containing z 
and z' and intersecting the unit circle at right angles. This circle degenerates to a straight line when the two points 
lie on the same diameter Any geodesic uniquely defines the reflection through it. Reflections are orientation 
reversing, one representative being the complex conjugation: k ■ z = z. 

We distinguish three different kinds of direct isometrics in V, according to which conjugacy class of the 
following one parameter subgroups it does belong: 
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Note that r^p■ z ~ e"^ z for 2 e X> and also, a^ • = tanh(i). The elements of A are sometimes called "boosts" 
[4]. The following theorem is called the Iwasawa decomposition (see [27]). 

Theorem 1. 

SU(1,1) = K- A-N 

The orbits of A converge to the same limit points b±i — ±1 on the unit circle when t — ^ ±cx). In particular the 
diameter (6_i, hi) is the orbit {x — tanh(f), t G R}. The orbits of A^ are the circles tangent to the unit circle at 
61. These circles are called howcycles with limit point bi (N is called the horocyclic group). The orbits of K are 
circles inside V (they coincide with Euclidean circles only when they are centered at the origin). 
Isometrics of the types K, A, N are respectively called elliptic, hyperbolic and parabolic. Elliptic isometries have 
one fixed point in V while hyperbolic isometries have two fixed points on the boundary dV and parabolic isome- 
tries have one fixed point on dV (hence "at infinity"). Hyperbolic isometries play the same role as translations 
in Euclidean space while parabolic isometries have no counterpart. There is however one important difference 
with Euclidean translations: two hyperbolic translations do not commute in general. The fundamental difference 
from this point of view between V and the Euclidean plane M^ is that the latter is itself an Abelian group while 
V ~ SU{1, 1)/5'0(2,K) is not a group. This makes its analysis, especially its Fourier analysis, harder and less 
intuitive. 

We can now define a (periodic) lattice in V and in 2? x M*. Let F be a discrete subgroup of SU{1, 1) such that 
the orbits of points in V under the action of F have no accumulation point in P. This is a Fuchsian group. To any 

o 

Fuchsian group we can associate a fundamental domain which is the closure, noted Fr, of an open set Fr C T) 
with the following properties [28]: 

(i) if 7 7^ Iff e F, then 7Fr n i^r = 0; 
(ii) U 7^r = v. 

Hence Fr generates a periodic tiling (or tessellation) of V. A fundamental domain need not be a compact subset 
of V (it may have vertices on the circle at infinity dV). When it does, F is called a cocompact Fuchsian group. 
In this case F contains no parabolic element, its area is finite and a fundamental domain can always be built as 
a polygon (the Dirichlet region, see [ ]). As mentioned in the introduction we identified in [' '] a family of 
Fuchsian groups, some of them compact, that naturally arose from the analysis of the symmetries in the spatial 
distribution of the photoreceptors in the retina. 

The following definition is just a translation to the hyperbolic plane of the definition of an Euclidean lattice. 

Definition 2. A lattice group ofD is a cocompact Fuchsian group which contains no elliptic element. 

The action of a lattice group has no fixed point, therefore the quotient surface V/V is a (compact) manifold 
and it is in fact a Riemann surface. A remarkable theorem states that any compact Riemann surface is isomorphic 
to a lattice fundamental domain of V if and only if it has genus 5 > 2 [28]. The case g — \ corresponds to 
lattices in the Euclidean plane (in this case there are three kinds of fundamental domains: rectangles, squares and 
hexagons). The simplest lattice in T), with genus 2, is generated by an octagon and will be studied in detail in 
Section 4. 

Given a lattice, we may ask what is the symmetry group of the fundamental domain Fr, identified with 
the quotient surface V/T. Indeed, this information will play a fundamental role in the subsequent bifurcation 
analysis. In the case of Euclidean lattice, the quotient M? /T is a torus T (genus one surface), and the group of 
automorphisms is "H xi T where Ti, is the holohedry of the lattice: Ti, = D2, D4 01 Dq for the rectangle, square 
and hexagonal lattices respectively. In the hyperbolic case the group of automorphisms of the surface is finite. In 
order to build this group we need first to introduce some additional definitions. 

Tilings of the hyperbolic plane can be generated by reflections through the edges of a triangle r with vertices 
P, Q, R and angles 7r/i?, n/m and n/n respectively, where £, m, n are integers such that l/i + 1/m + 1/n < 1 
[28]. 

Remember that reflections are orientation-reversing isometries. We note k, k' and k" the reflections thro ugh 
the edges PQ, QR and RP respectively (Figure 1). The group generated by these reflections contains an index 
2 Fuchsian subgroup A called a triangle group, which always contains elliptic elements because the product of 
the reflections through two adjacent edges of a polygon is elliptic with fixed point at the corresponding vertex. 
One easily shows that A is generated by the rotations of angles 2^/1, l-n jm and 27r/n around the vertices P, Q, 




Figure 1: The triangle t, also noted T{2, 3, 8). The values of I, m, and n are I = 8,m = 2 and n = 3. 
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Figure 2: Tesselation of the hyperbolic octagon with the triangle T(2, 3, 8), colored in purple in the plot. We 
define two points S and S. S is the center of the rotation ct by tt (mod F), see text in subsection 4.1. S* is the 
center of the rotation ahy tt (mod F), see text in subsection 4. 1 . 



R re spectively. A fundamental domain of A is the "quadrangle" F\ = t U kt [^ ]. Note that Fa ~ D/A is 
a sphere (genus surface) obtained by identifying the three edges of t. The subgroup of hyperbolic translations 
in A is a lattice group F, normal in A, whose fundamental domain is filled with copies of the basic tile r. The 
group of orientation-preserving automorphisms of Fp ~ V/T is therefore G — A/F. From the algebraic point of 
view, G is generated by three elements a, 6, c satisfy therelations a^ = 6™ = c" = 1 and a ■ h ■ c — 1. We say 
that G is an (^, tti, n) group. Taking account of orientation-reversing isometries, the full symmetry group of Fy is 
G* — Gyj kG = G X Z2{k). This is also a tiling group of Fr with tile r: the orbit G*t fills Fr and its elements 
can only intersect at their edges. 

Given a lattice, how to determine the groups G and G* ? The following theorem gives conditions for this, see 
[9]. 

Theorem 2. An (/, m, n) group G is the tiling rotation group of a compact Riemann surface of genus g if and only 
if its order satisfies the Riemann-Hurwitz relation 
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|G|=, n , 1 , 1, 
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Tables of triangle groups for surfaces of genus up to 13 can be found in [9]. 
These definitions extend naturally to SDP{2) ~ P x M* as follows. 

Definition 3. A lattice group ofDxR* is a subgroup of the fonn F x S where F is a lattice group acting in V 
and S is a non trivial discrete subgroup o/K*. 

Any discrete subgroup of R* is generated by a positive number a and can be further identified with Z: E — 
{a", n E Z}. A fundamental domain for F x S is a "box" Fr x [1, a]. 

2.3 Plane waves in SDP(2) 

Let us first recall the Euclidean setting for functions defined in M.^ (we could take any other M", n > 0). In this 
case every function of the form e^'''' where k G M"^ is a unit vector, is an eigenfunction of the Laplace operator in 

The fact that the eigenvalues do not depend upon the direction of the wave vector k reflects the rotational invariance 
of the Laplace operator Moreover if we take X ^ ia, a E R, then e'^'^ "■ is invariant under translations in M^ by 
any vector e satisfying the condition k • e = 2n7r where n E Z (it clearly does not depend upon the coordinate 
along the axis orthogonal to k). The functions e'"'^'' are elementary spatial waves in M.^. 

Now, given a > and a basis of unit vectors {ki , k2 , ka} of M.'^ we can define the translation group C spanned 
by e^, i = 1,2, 3, such that kj • Gj = 2Tr/aSij. Hence £ is a lattice group of R^. It defines a periodic tiling, 
the fundamental domain of which is a compact cell which we may identify with the quotient space R'^ /£ and 
which we can identify with a 3-torus. Any smooth enough function in R'^ which is invariant under the action 
of C can be expanded in a Fourier series of elementary spatial waves g»"("«ki+nk2+pk3)r^ m,n,p E 1. The 
Laplace operator in the space of square-integrable functions in R? j C is self-adjoint and its spectrum consists of 
real isolated eigenvalues with finite multiplicities. The multiplicity depends upon the holohedry of the lattice, 
which we defined in the previous section (the largest subgroup of 0(3) leaving invariant the lattice). There are 
finitely many holohedries (see [ ] for details). It follows from the above considerations that by restricting the 
analysis to classes of functions which are invariant under the action of a lattice group, one can apply standard 
techniques of equivariant bifurcation theory to assert the generic existence of branches of solutions of Euclidean 
invariant bifurcation problems, which are spatially periodic with respect to lattice groups and whose properties are 
largely determined by the holohedry of the lattice [ ],[ ]. Note also this was the approach of [ ] for the analysis 
of the occurence of visual hallucinations in the cortex. 

We wish to apply the same idea to bifurcation problems defined in SDP{2). For this we need to define 
elementary eigenfunctions of the Laplace-Beltrami operator such that spatially periodic functions (in a sense to be 
defined later) can be expanded in series of these elementary "waves" in SDP{2). In the sequel A will denote the 
Laplace-Beltrami operator in SDP{2) or, equivalently, in I? x M+. 

Let 6 be a point on the circle dD, which we may take equal to 6i = 1 after a suitable rotation. For z E T), 
we define the "inner product" (z, 6) as the algebraic distance to the origin of the (unique) horocycle based at h 
and passing through z. This distance is defined as the hyperbolic signed length of the segment OS, where ^ is the 



intersection point of the horocycle and the line (geodesic) Ob. Note that (z, b) does not depend on the position of z 
on the horocycle. In other words, (z, b) is invariant under the action of the one-parameter group N (see definition 
above). The "hyperbolic plane waves" 

ep^b(z) = e('''+5)(-.''>,peC, 
satisfy 

where A^i is defined in Corollary 2. These are the elementary eigenfunctions with which Helgason built a Fourier 
transform theory for the Poincare disc, see [ ]. It follows from Helgason's theory that any eigenfunction of Ap 
can be expressed as an integral over the boundary elements: 

Theorem 3. [24] Any eigenfunction of the operator — Ap admits a decomposition of the form 

e-p,b{z)dTp{b) 



IdD 

where Tp is a distribution defined on the circle dD and the eigenvalue is p^ 



Real eigenvalues — {p^ + | ) of Ap correspond to taking p real or p e iM. The latter case is irrelevant for our 
study as it corresponds to exponentially diverging eigenfunctions. Therefore the real spectrum of A is continuous 
and is bounded from above by —1/4. By using (9) we extend Theorem 3 to I? x M+: 

Corollary 3. (/) Let us note z — (zi, Z2, Z3) G I? x M+ and z = zi + iz2. The function 

V'p,M(z) = ep,fc(z)e^'°s/3iog.3 (13) 

satisfies the relation Atpp^t.p ~ ^{p^ + i + ^"^S P)^p,b,p- 
(ii) Any eigenfunction of A admits a decomposition of the form 

g»iog/3iog.3 f ep^t{z)dTp{b). (14) 

JdV 

From there we can extend the definition (and properties) of the Fourier transform in the hyperbolic plane given 
by Helgason [ ] to the space V x M+: 

Definition 4. Given a function f onV x M+, its Fourier transform is defined by 

f{p,b,(3)^ [ /(z)e_p,6(z)e-"°s'3iog.3rf2 (I5) 

JvxRt 

In the following we will look for solutions of bifurcation problems in 2? x M+, which are invariant under the 
action of a lattice group: (7, ^) • u{z, Z3) ~ u(7~^z, ■^"^^3) = u{z, Z3) for 7 £ F, ^ e S. This boils down to 
looking for the problem restricted to a fundamental domain with suitable boundary conditions imposed by the F- 
periodicity, or, equivalently, to looking for the solutions of the problem projected onto the orbit space V/T x M+/S 
(which inherits a Riemannian structure from V x M+). Because the fundamental domain is compact, it follows 
from general spectral theory that —A is self-adjoint, non negative and has compact resolvent in L^(I?/F x M+/S) 
[10]. Hence its spectrum consists of real positive and isolated eigenvalues of finite multiplicity. 

Coming back to Theorem 3, we observe that those eigenvalues A of —Ax> which correspond to F-invariant 
eigenfunctions, must have p E R 01 p E iR. The case p real corresponds to the Euclidean situation of planar 
waves with a given wave number, the role of which is played by p in V. In this case the eigenvalues of — Ap 
satisfy 1/4 < A. On the other hand there is no Euclidean equivalent of the case p G iM., for which the eigenvalues 
< A < 1/4 are in finite number. It turns out that such "exceptional" eigenvalues do not occur for "simple" 
groups such as the octagonal group to be considered in more details in the Section 4. This follows from formulas 
which give lower bounds for these eigenvalues. Let us give two examples of such estimates (derived by Buser 
[10], see also [27]): (i) if 5' is the genus of the surface V/T, there are at most 3g — 2 exceptional eigenvalues; (ii) 
if d is the diameter of the fundamental domain, then the smallest (non zero) eigenvalue is bounded from below by 
(47r sinh|)"l 



Suppose now that the eigenfunction in Theorem 3 is F-periodic. Then the distribution Tp satisfies the following 
equivariance relation [ ]. Let 7(6*) denote the image of 6* e dT> under the action of 7 e F. Then 

As observed by [ ], this condition is not compatible with Tp being a "nice" function. In fact, not only does there 
not exist any explicit formula for these eigenfunctions, but their approximate computation is itself an uneasy task. 
We shall come back to this point in subsequent sections. 

3 Bifurcation of patterns in SDP(2) 

We now consider again equation (1), which we set in 2? x M+ by the change of coordinates (6). Assuming I — Q 
(no external input) and the invariance hypothesis (3) for the connectivity function w, and after a choice of time 
scale such that a = 1, the equation reads 

dV 

^-V + ^iw*V + R{V) (16) 

OT 

where: 

• ^i = s"(o), 

• w *V denotes the convolution product /p>,]g+ if (z, z')y (z') rfz' (with w(z, z') = f{d{z, z'))), 

• R{V) stands for the remainder terms in the integral part of (3). This implies -R'(O) = 0. 
It is further assumed that / is a "Mexican hat" function, typically of the form 

1 — ^^, 1 — j:^ 



/(^) 






2 



where cti < 0-2 and < 1. 

Let us look at the linear stability of the trivial solution of (16) against perturbations in the form of hyperbolic 
waves (13) with p E M.. This comes back to looking for cr's such that 

cr = — 1 + /iw) 

where w is the hyperbolic Fourier transform of w as defined in Definition 15. The numerical calculation shows 
that for each value of p and /3, there exists a value p,{p, (3) such that if /i < p,{p, (3) then all cr's are negative, 
while CT = at ^ = p{p,f3). The "neutral stability surface" defined by p{p, f3) is typically convex and reaches 
a minimum pc at some values pd^c- Therefore when p < pc the trivial state y = is stable against such 
perturbations while it becomes marginally stable when p = pc with critical modes ipp^^b,i3a^ for any b E S^ 
(rotational invariance). Therefore a bifurcation takes place at this critical value. 

The situation is absolutely similar if instead of equation 16 we consider systems of PDEs in 2? x M+ with 
pattern selection behavior and with t/(l, 1) x i?+ invariance. A paradigm for such systems is the "Laplace- 
Beltrami" version of Swift-Hohenberg equation 

dt ' * 

with A as in (9). 

It is unconceivable to solve the bifurcation problem at this level of generality, because the fact that the spectrum 
is continuous plus that each eigenvalue a has an infinite multiplicity (indifference to b) makes impossible the use 
of the classical tools of bifurcation theory. As in the Euclidean case of pattern formation, we therefore want to look 
for solutions in the restricted class of patterns which are spatially periodic. In the present framework, this means 
looking for bifurcating patterns which are invariant under the action of a lattice group F x Z+ in SU{1, 1) x Rf. 
There is however an immediate big difference with the Euclidean case. While in the latter any critical wave 
number ac can be associated with a periodic lattice (of period 27r/ac), in the hyperbolic case not every value of 
Pc can be associated with a lattice in V. More precisely it is not generic for the Laplace-Beltrami operator in V 



to possess eigenfunctions, as defined in Theorem 3, which are invariant under a lattice group. We can therefore 
look for the bifurcation of spatially periodic solutions associated with a given lattice, but these patterns will not in 
general correspond to the most unstable perturbations unless the parameters in the equations are tuned so that it 
happens this way. The question of the observability of such patterns is therefore completely open. 

We henceforth look for patterns in 2? x M+ which are invariant under a lattice F in 2? and which are periodic, 
with period 27r//3c, in M+. This boils down to looking for solutions in the space L^{'D/T x M+//3cZ+). Note that 
M+//3cZ+ ~ S^. With a suitable inner product this space admits an orthonormal Hilbert basis which is made of 
functions of the form 

*(z)e"'°s(/9c)iog(^3)^ zeP, neN 

where Vf are the eigenfunctions of A in L?{T>/V). As we mentionned in the previous section, these eigenfunctions 
are not known explicitely. By restricting the "neutral stability surface" /i(p, /3) to those values which correspond 
to eigenfunctions with F x /3cZ+ periodicity, we obtain a discrete set of points on this surface with one minimum 
/io associated with a value po of P ^nd /3o of /3. In general this minimum is unique. Moreover the multiplicity of 
the eigenvalue is now finite and this eigenvalue is semi-simple. Let us call X the eigenspace associated with the 
eigenvalue (therefore X is the kernel of the critical linear operator). 

The full symmetry group ofD/T x S^ is equal to G* x 0(2) where G* is the (finite) group of automorphisms 
in C/(l, 1) of the Riemann surface V/T (notation of Section 2.2) and 0(2) is the symmetry group of the circle 
(generated by 5"^ and by reflection across a diameter). The equation restricted to this class of F x /3cZ+-periodic 
patterns is invariant under the action of G* x 0(2). We can therefore apply an equivariant Lyapunov-Schmidt 
reduction to this bifurcation problem [12], leading to a bifurcation equation in X 

/(x,/i)-0, x^X (17) 

where / : X x M — > X is smooth, /(O, 0) = 0, dxf{Q, 0) is not invertible and /(•, /i) commutes with the action 
ofG* X 0{2)mX. 

Now the methods of equivariant bifurcation theory can be applied to (17). In particular we can apply the 
Equivariant Branching lemma (see [21] for a detailed exposition): 

Theorem 4. Suppose the action of G* x 0(2) is absolutely irreducible in X (i.e. real equivariant linear maps 
in X are scalar multiple of the identity). Let H be an isotropy subgroup of G* x 0(2) such that the subspace 
X^ — {x E X \ H ■ X — x} is one dimensional. Then generically a branch of solutions of (17) bifurcates in X^ . 
The conjugacy class of H (or isotropy type) is called "symmetry breaking". 

Let us briefly recall the meaning of this theorem. By equivariance of /, any subspace of X defined as X^ , H a 
(closed) subgroup of G* x 0(2), is invariant under /. By the irreducibility assumption, if H = G* x 0(2), then 
{x £ X \ H ■ X ^ x} — {0}. Therefore /(0,/i) = for all fi. Now the assumption of absolute irreducibility 
implies that dxf{0, /i) = a{fi)Idx where a is a smooth real function such that a(0) = and generically a' (0) =^ 0. 
It follows that if now iJ is a subgroup such that dim X^ — 1, then equation ( 1 7) restricted to this subspace reduces 
to a scalar equation = a'(0)/ia; + xh{x, ji) (with /i(0, 0) = 0), which has a branch of non trivial solutions by the 
implicit function theorem. 

Remark 1: the word "generically" can be interpreted as follows: the result will fail only if additional degeneracies 
are introduced in the equations. See [ ] and [ ] for a rigorous definition and proof 

Remark 2: the assumption of absolute irreducibility is itself generic (in the above stated sense) for one-parameter 
steady-state bifurcation problems. A given irreducible representation of a compact or finite group need not be 
absolutely irreducible, this fact has to be proven. 

Remark 3: Theorem 4 does not necessarily give an account of all possible branches of solutions of (17), see [ I 2]. 
It gives nevertheless a large set of generic ones. To go further it is necessary to compute the equivariant structure 
of /, or at least of its Taylor expansion to a sufficient order. The same is true if one wants to determine the stability 
of the bifurcated solutions, within the class of F x 5*^ periodic solutions of the initial evolution equation. 

This theorem, together with the knowledge of the lattices and the (absolutely) irreducible representations of 
the groups G*, gives us a mean to classify the periodic patterns which can occur in P x M+. By analogy with 
the Euclidean case (bifurcation of spatially periodic solutions in the Euclidean space), we call H-planforms the 
solutions of a C/(l, 1) (resp. GL(2M)) invariant bifurcation problem in T) (resp. T) x M+), which are invariant by 
a lattice group F (resp. F x S^). 

Being interested here in the classification of solutions rather than in their actual computation for a specific 
equation, all remains to do is to determine the absolutely real irreducible representations of the group G* x 0(2) 
and the computation of the dimensions of the subspaces X^). For this purpose we can get rid of the S^ component 
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of the domain of periodicity. Indeed let H = Hi x H2 he an isotropy subgroup for the representation R of 
G* X 0(2) acting in X. Then X = V <E)W and R = S <^T, where S is an iiTeducible representation G* in V 
and T is an irreducible representation of 0{2) in 14^ [37], and therefore Hi acts in V and i?2 acts in W. Now we 
have the following lemma, the proof of which is straightforward: 

Lemma 1. dim(X^) = 1 if and only if dim{V^') = dim{W^') = 1. 

Now, the irreducible real representations of 0{2) are well-known: they are either one dimensional (in which 
case every point is rotationally invariant) or two-dimensional, and in the latter case the only possible one dimen- 
sional subspaces W^'^ are the reflection symmetry axes in M^ (which are all equivalent under rotations in 0(2)). 
It follows that the classification is essentially obtained from the classification of the isotropy subgroups Hi of the 
irreducible representations of G* . 

Once the irreducible representations are known, this can be achieved by applying the "trace formula" [21, 12]: 

Proposition 3. Let H be a subgroup ofG* acting in a space V by a representation p : G* ^ Aut[V), then 

dMV") - r^ E tr(p(/^)) (18) 

Note that tr(p) is the character of the representation p (a homomorphism G* — > C). What is really needed to 
apply the Equivaraint Branching Lemma is therefore the character table of the representations. In the next section 
we investigate this classification in the case when the lattice is the regular octogonal group. 

4 A case study: the octagonal lattice 

According to the comment following Lemma 1, in all of this section we only consider the classification of isotropy 
subgroups satisfying the conditions of the equivaraint branching lemma in the Poincare disc T). 

4.1 The octagonal lattice and its symmetries 

Among all lattices in the hyperbolic plane, the octagonal lattice is the simplest one. As before we use the Poincare 
disc representation of the hyperbolic plane. Then the octagonal lattice group F is generated by the following four 
hyperbolic translations (boosts), see [4]: 

/ 1 + ^/2 ^2 + 2^/2 \ 

and gj = 7'j7r/45o''-j7r/4i j = 1, 2, 3, where r^ indicates the rotation of angle ip around the origin in V. The 
fundamental domain of the lattice is a regular octagon O as shown in Figure. The opposite sides of the octagon 
are identified by periodicity, so that the corresponding quotient surface V/T is isomorphic to a "double dough- 
nut" (genus two surface) [ ]. Note that the same octagon is also the fundamental domain of another group, not 
isomorphic to F, obtained by identifying not the opposite sides but pairs of sides as indicated in Figure. This is 
called the Gutzwiller octagon. A procedure of classification of the lattices using graphs is presented in [ ]. For 
us however there is no difference between the two kinds of octagons because we are really interested in the full 
symmetry group of the pattern generated by F, which includes the rotations rjT/4, j = 1, • • • ,8, and therefore 
the boosts rj^/2gQ^, <?i?'-7r/2 and their conjugates by the rotation r^, which are precisely the generators of the 
Gutzwiller lattice group. 

We now determine what is the full symmetry group G* of the octagonal lattice, or equivalently, of the surface 
V/r. Clearly the symmetry group of the octagon itself is part of it. This is the dihedral group Dg, generated by the 
rotation r^^/^ and by the reflection k through the real axis, but there is more. We have seen in Section 2.2 that the 
group G* = A/F, A being the triangle group generated by reflections through the edges of a triangle r which tiles 
(by the action of A/F) the surface V/T. The smallest triangle (up to symmetry) with these properties is the one 
shown in Figure 1. It has angles 7r/8, tt/2 and 7r/3 at vertices P = O (the center of V), Q, R respectively, and 
its area is, by Gauss-Bonnet formula, equal to 7r/24. There are exactly 96 copies of r filling the octagon, hence 
|G*| = 96. The index two subgroup G of orientation-preserving transformations in G* has therefore 48 elements. 
In [8] it has been found that G ~ GL{2, 3), the group of invertible 2x2 matrices over the 3 elements field Z3. In 
summary: 
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Proposition 4. The full symmetry group G* ofD/T is G U kG where G ~ GL{2, 3) has 48 elements. 

The isomorphism between GL{2, 3) and G can be buih as follows. We use the notation Z3 = {0, 1, 2} and we 
call p the rotation by 7r/4 centered at P (mod F), a the rotation by n centered at Q (mod F) and e the rotation by 
2tt/3 centered at R (mod F). In the notations of Section 2.2, a — a, b ~ e, c — p, and pae = 1. Then we can take 



P 



2 
2 2 



2 
1 



2 1 
2 



— ^^ = Id and pcre = /d. Note that p'' = —Id where Id is the 



since these matrices satisfy the conditions p^ = a" = e 
identity matrix. We shall subsequently use this notation. The group GL{2, 3), therefore the group G, is made of 
8 conjugacy classes which we list in Table 1, indicating one representative, the number of elements in each class 
and their order. This result is classical and can be found, e.g., in [ ]. 



representative 


Id 


P 


P' 


-Id 


P' 


a 


e 


— e 


order 


1 


8 


4 


2 


8 


2 


3 


6 


# elements 


1 


6 


6 


1 


6 


12 


8 


8 



Table 1: Conjugacy classes of G ~ GL{2, 3) 

We now turn to the full symmetry group G* which is generated by G and k, the reflection through the real axis 
in T> and which maps the octagon O to itself. We write k' — pK the reflection through the side PR of the triangle 
T. Note that (i) k' preserves also O, (ii) k" — en' — an is the reflection through the third side QR. 

In what follows we rely on the group algebra software GAP [ ] . For this we have first identified a presentation 
for G ~ GL{2, 3) considered as an abstract group, then a presentation for G* . The presentation for GL{2, 3) can 
be obtained with the command "P := Presentation ViaCosetTable(GL(2,3))" and the relations are shown with the 
command "TzPrintRelators(P)": 

Lemma 2. (i) As an abstract group, G is presented with two generators a and b and three relations a? = 1, 
b^ ~ 1 and (abab^^ab^^)^ — 1. (ii) As an abstract group, G* is presented with three generators a, b and c and 
six relations: the three relations for G plus the three relations (? = 1, [cd)^ = 1 and (cb)'^ = 1. 
(Hi) These abstract elements can be identified with automorphisms of D/T as follows: a = a, b = e and c — n" . 

Applying the above lemma we find with GAP that the 96 elements group G* has 13 conjugacy classes which 
are listed in table 2 for direct isometrics and in table 3 for isometrics which reverse orientation. GAP gives rep- 
resentatives of the conjugacy classes in the abstract presentation, which in general have complicated expressions. 
In some cases we have chosen other representatives, using in particular the 8-fold generator. To simplify some 
expressions in the tables 2 to 5 we also use the notations 

CT = ecre^^, 5 = p^ap^^ , 

where a is the rotation by tt centered at S (mod F) and a is the rotation by tt centered at S (mod F), see figure 2. 

We shall also need in Section 4.3 the list of subgroups of G* together with their decomposition in conjugacy 
classes (in G*) in order to apply the trace formula (18). Here again we rely on GAP to obtain the necessary 
informations. 

Then representatives of each class are determined by inspection. These datas are listed in tables 4 (subgroups 
of G) and 5 (subgroups containing orientation reversing elements). The subgroups are listed up to conjugacy in 
G*, the subgroups of order two are not listed. The rationale for the notations is as follows: 



class number 


1 


2 


3 


4 


5 


6 


7 


representative 


Id 


P 


P' 


-Id 


(T 


e 


—t 


order 


1 


8 


4 


2 


2 


3 


6 


# elements 


1 


12 


6 


1 


12 


8 


8 



Table 2: Conjugacy classes of G*, orientation preserving transformations 
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class number 


8 


9 


10 


11 


12 


13 


representative 


K 


k' 


aK 


pan 


CK 


—en 


order 


2 


2 


8 


4 


12 


12 


# elements 


8 


8 


12 


12 


4 


4 



Table 3: Conjugacy classes of G* , orientation reversing transformations 



Subgroup 


Order 


Generators 


Subclasses: representatives (# elements) 


Go~S'L(2,3) 


24 


< p^, e > 


{Id (1), -Id (1), p2 (6), e (8), -e (8)} 


D^ 


16 


< p, CT > 


{Id (1), -/d (1), p (4), p2 (6), CT (4)} 


De 


12 


< -e, CT > 


{/d(l),-/d(l),a(6),e(2),~e(2)} 


Cs 


8 


<P> 


{/d(l),-Jd(l),p(4),p^(2)} 


Qs 


8 


< p'^, crp^cr > 


{Id (1), -Id (I), p^ (6)} 


Di 


8 


<p^,a> 


{/rf(l), -/d(l),p2(2),a(4)} 


Ce 


6 


< -e > 


{M(l), -W(l), e(2), -e(2)} 


Dz 


6 


< e, ? > 


{/d(l),e(2),5(3)} 


Ci 


4 


< p^ > 


{M(l),-/d(l),p^(2)} 


D2 


4 


< -Id, (T > 


{M(l), -W(l), fT(2)} 


Cs 


3 


< e > 


{/d(l),e(2)} 


C2 


2 


<-Id> 


{Id(l),-Id(l)} 


C2 


2 


< a > 


{Id (11 a (I)} 



Table 4: Subgroups of G c G* (up to conjugacy). The last column provides datas about their conjugacy subclasses 
(inG*). 

• Go is an index 2 subgroup of G. Seen as a subgroup of GL{2, 3) it is SL{2, 3), the subgroup of determinant 
1 matrices. It contains no order 2 elements except —Id and no order 8 elements. 

• G„, G„, G^, denote order n cyclic groups. The notation G„ is standard for the n-fold rotation group centered 
at the origin. 

• Dn denotes a group isomorphic to the dihedral group of order 2n, generated by an n-fold rotation and a 
reflection. Hence Dg is the symmetry group of the octagon. The notation £)„ is used for a 2n element group 
which has an n-fold rotation and a 2-fold rotation as generators. For example Ds =< p, a >, and one can 
verify that apa^^ = p^, which makes Dg a quasidihedral group (see [22]). 

• Qs is a usual notation for the 8 elements quaternionic group. 

• The notation HnK indicates a group generated by the group i/„ and n. Same thing if replacing n by k! . For 
example Gsk' is the 6 elements group generated by G3 and k! . 



4.2 The irreducible representations of G* 

There are 13 conjugacy classes and therefore we know there are 13 complex irreducible representations of G* , the 
characters of which will be denoted Xj, j — 1, ..., 13. The character table, as computed by GAP, is shown in table 
6. 

The character of the identity is equal to the dimension of the corresponding representation. It follows from 
table 6 that there are 4 irreducible representations of dimension 1, 2 of dimension 2, 4 of dimension 3 and 3 
of dimension 4. In the following we shall denote the irreducible representations by their character: Xj is the 
representation with this character. 

Lemma 3. All irreduclible representations ofG* listed in table 6 are real absolutely irreducible. 
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Subgroup 


Order 


Generators 


Subclasses: representatives (# elements) 


Gok 


48 


< Go, K > 


Go U {k (6), paK (2), en (8), -en (8)} 


Gqk' 


48 


< Go, k' > 


Go U {k' (12), an (12)} 


Dsn 


32 


< Ds, K > 


Ds U {k (6), k' (4), CTK (4), paK (2) } 


i?6K' 


24 


< i:'6, k' > 


De U {k' (6), en (2), -en (2), paK (2)} 


Csk (=Ds) 


16 


< Gg, K > 


Gg U {k (4), k' (4)} 


CL 


16 


< p^CT, K > 


Gg U {k (2), /3CTK (2), an (4)} 


Qsk 


16 


< Qs, K> 


gg U {k (6), pCTK (2)} 


Qsk' 


16 


< Qs, k' > 


gg U {k' (4), CTK (4)} 


^4k 


16 


< D4, K> 


L»4 U {k (4), CTK (4)} 


i^4.' 


16 


< Di, k' > 


Di U {k (2), k' (4), pCTK (2)} 


C(2 


12 


<eK> 


Ge U {eK (2), -ek (2), pai^ (2)} 


C6k' 


12 


< Cq,k' > 


Ge U {«:' (6)} 


r" 


8 


<(JK> 


G4 U {ctk (4)} 


C4K (= i^4) 


8 


<Ci,K> 


G4 U {k (4)} 


C^4k' 


8 


<Ci,K' > 


G4 U W (4)} 


D2. 


8 


< D2,K> 


D2 U {k (2), k' (2)} 


CL 


8 


<C'i,K> 


G^ U {p' (2), K (2)} 


CL' 


8 


<C'i,K' > 


Ci U {a (2), k' (2)} 


Csk' 


6 


<C3,k'> 


G3 U {/c' (3)} 


c. 


4 


< p(7K > 


{Id (1), -/rf (1), pCTK (2)} 


C2n 


4 


< -Id, K > 


{Id (1), -Id (I), K (2)} 


C2k' 


4 


< -Id, k' > 


{/d(l), -/d(l), k'(2)} 


C2k 


4 


< a,K > 


{/d(l),(T(l),K(l),«;'(l)} 


CL 


4 


<a,K> 


{M(1),S?(1), K (!),«;' (1)} 


Ci, 


2 


< K> 


{Id(l),K(l)} 


Cik' 


2 


< k' > 


{Id(l),K'(l)} 



Table 5: Subgroups of G*, not in G (up to conjugacy). The last column provides datas about their conjugacy 
subclasses (in G*). G4 is a subgroup conjugate to G4 with generator (p^cr)^. 



Class # 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


Representative 


Id 


P 


P 


-Id 


a 


e 


— e 


K 


k' 


(TK 


pan 


en 


-EK 


Xi 


1 


1 


1 


1 


1 






1 


1 


1 


1 






X2 


1 


-1 


1 


1 


-1 






1 


-1 


-1 


1 






X3 


1 


-1 


1 


1 


-1 






-1 


1 


1 


-1 


-1 


-1 


Xi 


1 


1 


1 


1 


1 






-1 


-1 


-1 


-1 


-1 


-1 


X5 


2 





2 


2 





-1 


-1 


-2 








.2 






X6 


2 





2 


2 





-1 


-1 


2 








2 


-1 


-1 


X7 


3 


1 


-1 


3 


-1 








-1 


-1 


1 


3 








X8 


3 


1 


-1 


3 


-1 








1 


1 


-1 


-3 








X9 


3 


-1 


-1 


3 


1 








1 


-1 


1 


-3 








Xio 


3 


-1 


-1 


3 


1 








-1 


1 


-1 


3 








Xii 


4 








-4 





-2 


2 




















X12 


4 








-4 





1 


-1 














V3 


-V3 


Xl3 


4 








-4 





1 


-1 














-V3 


V3 



Table 6: Irreducible characters of G* 
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Proof. This is clear for the one dimensional representations whose characters are real. 

For the two dimensional representations, let us consider the dihedral subgroup D-^ generated by the 3-fold symme- 
try e and the reflection k' . The representation of I?3 in either representations planes of xs and xe have characters 
Xj(e) = — 1 and Xj{^') = (j = 5 or 6). These are the characters of the 2D irreducible representation of D^, 
which is absolutely irreducible, and therefore the representations Xb and xe of G* are also absolutely irreducible. 
Indeed if any real linear map which commutes with the elements of a subgroup is a scalar multiple of the identity, 
this is a fortiori true for the maps which commute with the full group. 

For the three dimensional representations X7 to xio, let us first remark that if we write C2 = {Id, —Id}, then 
G/C2 — O, the octahedral group. Its subgroup T (tetrahedral group) can easily be identified with the 12 elements 
group generated by the "pairs" {Id, —Id}, {e, —e} and {p^, —p^}- Now we consider the representation of G 
defined by the action of Xj restricted to G (for each 3D Xj)- One can check easily from the character table that it 
projects onto a representation of G* /C2, the character of which is given by the value of Xj on the corresponding 
conjugacy classes, and in particular the character for the representation of the group T is given, for any j = 7 to 
10, by Xj{{Id, —Id}) — 3, Xj{{^j ^^}) — and Xj({p^, —p^}) = — 1- But this is the character of the irreducible 
representation of T [ ], which is absolutely irreducible (natural action of T in M^). Hence the three dimensional 
representations of G* are absolutely irreducible by the same argument as above. 

It remains to prove the result for the four dimensional representations xii, X12 and xi3- F'or this we consider the 
action of the group Dg generated by p and k, as defined by either one of these 4D irreducible representations of 
G* . We observe from the character table that in all cases, the character of this action is x(p) = 0, xip^) = 0, 
x{—Id) = —4, x{p^) — (p and p^ are conjugate in G*), and x('*) — x(^') — 0- We can determine the isotypic 
decomposition for this action of Dg, from these character values. The character tables of the four one dimensional 
and three two dimensional irreducible representations of Dg can be computed easily either by hand (see [3 1 ] for 
the method) or using a computer group algebra software like GAP. For all one dimensional characters the value at 
—Id is 1, while for all two dimensional characters, the value at —Id is —2. Since x{—Id) — —4, it is therefore 
not possible to have one dimensional representations in this isotypic decomposition. It must therefore be the sum 
of two representations of dimension 2. Moreover, since x{p) = xip'^) — x{p^) — 0, it can't be twice the same 
representation. In fact it must be the sum of the representations whose character values at p are y/2 and — \/2 
respectively. Now, these representations are absolutely irreducible (well-know fact which is straightforward to 
check), hence any ZJg-equivariant matrix which commutes with this action decomposes into a direct sum of two 
scalar 2x2 matrices A/2 and /i/2 where A and p, are real. But the representation of G* is irreducible, hence X — p, 
which proves that it is also absolutely irreducible. D 

4.3 The octagonal H-planforms 

We can now apply Lemma 1 in order to determine the H-planforms for the octagonal lattice. 

Theorem 5. The irreducible representations ofG* admit H-planforms with the following isotropy types: 

• Xi: G*; 

• X2- Gok,- 

• X3- Gok',- 

. X4.-G~GL(2,3); 

• X5- Dg, ggK'/ 

• X7-' C'g^, (7^2, C4K'; 

• Xs.- Gsk, Cqk,', D2k; 

• X9- De, Dak; 

• xio- Dqk', Dak-; 

• Xii-' G2K, C'2^; 
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• Xl2-' D3, C'iK', C2K, C2K.; 

• Xl3-' D3, Csk.', C2K, C2K,; 

Proof. For the one dimensional representations of G* this is straightforward: each element whose character image 
is +1 belongs to the isotropy group. The result follows therefore directly from the character table and list of 
subgroups of G* . For the higher dimensional irreducible representations we need to find those isotropy subgroups 
i/ such that (see (18)): 

l = dim(0 = ^^X,(/^) 
' ' heH 

where Xj denotes the character of the j-th irreducible representation. This can be done in a systematic way by 
using the character table 6 and applying the datas on subgroups and their conjugacy classes listed in 4 and 5. The 
calculations are cumbersome but can be slightly simplified by noting that if H C H' and dim(y^) = (a case 
which occurs many times), then not only H is not symmetry breaking but also H', since H C H' ^ V^ C V^ . 
The following lemma is also useful as it eliminates most candidates in the case of 4D representations: 

Lemma 4. If a subgroup H contains —Id, then for j — 11, 12 or 13, one has X]/ieij XjW — 0- 

Proof of the lemma. In all three cases the result follows from the relations: (i) Xj i^Id) = ^Xj {Id), (ii) x{^^) = 
— x(e) and x(— ck) = — x(eK), (iii) Xj(s) = for all s which is not conjugate to one in (i) or (ii). 

n 

For the lower dimensional representations it is possible to reduce the problem to known situations and to 
provide bifurcation diagrams without any further calculations. The next theorem provides these informations. 
Stability of the solutions has to be understood here with respect to perturbations with the same octagonal period- 
icity in V and under the condition that, in L^(I?/r), the corresponding representation corresponds to the "most 
unstable" modes ("neutral modes" at bifurcation). 

Theorem 6. For the one and two dimensional representations, the generic bifurcation diagrams have the following 
properties: 

• xi- transcritical branch, exchange of stability principle holds. 

• X2> X3 i^nd Xi- pictchfork bifurcation, exchange of stability principle holds. 

• X5- same as bifurcation with hexagonal symmetry in the plane, see Figure 3. 

• xe- same as bifurcation with triangular symmetry in the plane. In particular H-planforms are always 
unstable on both sides of the bifurcation point unless the subcritical branch bends back sufficiently near the 
bifurcation point (see Figure 4). 

Proof, (i) For the one dimensional representations, this follows from classical bifurcation theory: in xi there 
is no symmetry breaking, hence generically the bifurcation is of transcritical type and the trivial and bifurcated 
solutions exchange stability at the bifurcation point. In the three other cases, a symmetry exists which acts by 
reversing direction on the axis as can be seen from Table 6. For example in X2 this can be taken as a (but also k' 
does the same thing). Hence the bifurcation is of pitchfork type and exchange of stability holds, 
(ii) For X5, note that the subgroup Qg acts trivially on any point of this plane (dim(y'^**) = 2). In fact Qg 
is the isotropy group of the principal stratum in this group action. Now, G ~ GL{2, 3) = Qs tK D^, hence 
G* /Qs — Ds K Z2 — Dq, the symmetry group of an hexagon. This group action is isomorphic to the natural 
action of Dg in the plane. It follows that the problem reduces in this case to a bifurcation problem with the action 
of Dg in the plane, see [ ] for details. 

(iii) In the case of xe. the maximal subgroup which keeps every point in the plane fixed is the 16 element group 
Qsk.- It follows that the problem reduces to a bifurcation problem in the plane with symmetry G* /Qsk — Ds- 
Details on this bifurcation can be found in [2 1]. D 

Remark 1: A similar reduction can be made with the 3 dimensional representations. Indeed it can be seen that 
the principal isotropy type (which keeps all points in the three dimensional representation space fixed) is C4 for 
X7 and xio, and C2 for xs and X9- In the first case, this leads to reducing the problem to one with G* /C'^ ~ O 
symmetry, where O is the 24 elements group of direct symmetries (rotations) of a cube. However there are two 
irreducible representations of dimension 3 of O [ ], and it turns out that X7 corresponds to one of them (the 
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Figure 3: Bifurcation diagram for the case Xb- Dotted lines: unstable branches, . 
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Figure 4: Bifurcation diagram for the case xa- Dotted lines: unstable branches. 

"natural" action of O in M^) while xio corresponds to the other representation. This explains why there are 3 
types of H-planforms for X7 ^nd only 2 for xio- Similarly, the principal isotropy type for xs, ^nd xg is the two 
element group C2, and G* /C2 ~ O x Z2. Then the same remark holds for these cases as for the previous ones. 

Remark 2: In the 4 dimensional cases, the principal isotropy type is the trivial group, hence no reduction can 
be made. Bifurcation in this case (and in the 3 dimensional cases as well) will be the subject of a forthcoming 
paper. 

5 Computing the H-planforms 

It follows from the definition that H-planforms are eigenfunctions of the Laplace-Beltrami operator in T) which 
satisfy certain isotropy conditions: (i) being invariant under a lattice group F and (ii) being invariant under the 
action of an isotropy subgroup of the symmetry group of the fundamental domain V/T (mod F). Therefore in 
order to exhibit H-planforms, we need first to compute eigenvalues and eigenfunctions of A in T), and second to 
find those eigenfunctions which satisfy the desired isotropy conditions. In this section we tackle this question in 
the case where the lattice has the regular octagon as a fundamental domain. 

Over the past decades, computing the eigenmodes of the Laplace-Beltrami operator on compact manifolds has 
received much interest from physicists. The main applications are certainly in quantum chaos [-r, 2, 3, 35, 13] and 
in cosmology [26, 14, 30]. 

To our knowledge, the interest in such computation was sparkled by the study of classical and quantum me- 
chanics on surfaces of constant negative curvature, and the connections between them (for an overview on the 
subject see [4]). To be more precise, quantum chaology can be defined as the study of the semiclassical behaviour 
characteristic of systems whose classical motion exhibits chaos, for example the classical free motion of a mass 
point on a compact surface of constant negative curvature (as it is the most chaotic possible). In [ , "•, 1 ^], authors 
studied the time-independent Schroedinger equation on the compact Riemannian surface of constant curvature -1 
and genus 2, which is topologically equivalent to the regular octagon with four periodic boundary conditions. This 
is the same as solving the eigenvalue problem for F invariant eigenmodes in T). The first computations have been 
performed using the finite element method on "desymmetrised" domains of the hyperbolic octagon with a mixture 
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of Dirichlet and Neumann boundary conditions [4, 35] . We explain the procedure of desymmetrisation in the next 
subsection. Aurich and Steiner in [ ] were the first to compute the eigenmodes on the whole octagon with periodic 
boundary conditions. They began with the finite element method of type P2 and were able to exhibit the first 100 
eigenvalues. In [3], the same authors used the direct boundary-element method on an asymmetric octagon to reach 
the 20 000th eigenvalue. 

There is also a strong interest of cosmologists for ringing the eigenmodes of the Laplace-Beltrami operator 
on compact surfaces. Indeed this is necessary in order to evaluate the cosmic microwave background anisotropy 
in multiply-connected compact cosmological models. For some models, this computation is performed on a 
compact hyperbolic 3-space called the Thurston manifold, and Inoue computed the first eigenmodes of Thurston 
space such that each corresponding eigenvalue A satisfies A < 10 with the direct boundary-element method [2()]. 
For 3-dimensional spherical spaces, several methods have been proposed: the "ghosts method" [i-t], the averaging 
method and the projection method. All these methods are explained and summarized in [30]. 

Our aim is different in that we do not want to compute all the eigenvalues of the Laplace-Beltrami operator, but 
instead to calculate the H-planforms with the isotropy types listed in theorem 5. The methods of numerical compu- 
tation are however similar, and one question is to choose the method best suited to our goal. For the H-planforms 
associated to irreductible representations of dimension 1 (i.e. for (xi)i=i--4)' we use a desymmetrization of the 
octagon with a reformulation of the boundary conditions. For H-planforms associated with irreductible represen- 
tations of dimension > 2, the desymmetrization of the octagon is also possible but much more complicated as 
noticed by Balazs and Voros [ ]. This will therefore be the subject of another paper Here we only identify some 
H-planforms of specific isotropy types. In order to find these H-planforms, we use the finite-element method with 
periodic boundary conditions. This choice is dictated by the fact that this method will allow us to compute all the 
first n eigenmodes and among all these we will indentify those which correspond to a given isotropy group. As 
explained before, if we have used the direct boundary-element method, we would have reached any eigenmode 
but the search for H-planforms would also have became more random. Indeed, each iteration of this method gives 
only one eigenmode while the finite-element method provides n eigenmodes depending on the precision of the 
discretization. This is why we prefer to use this last method in order to find some H-planforms associated with 
irreductible representations of dimension > 2, although it is quite more complicated to implement because of the 
periodic boundary conditions. 

5.1 Desymmetrization of the octagon 

We have already seen that the fundamental domain T(2, 3, 8) of the group G* generates a tiling of the octagon. 
Desymmetrization consists in separating the individual solutions according to the symmetry classes of G* . This 
entails solving the eigenvalue problem in certain irreducible subregions of the fundamental domain, such as 
r(2, 3, 8), using special boundary conditions for these subregions. In effect the periodicity conditions in the 
original domain (octagon) may produce Dirichlet or Neumann conditions on the boundaries of these subregions. 
The symmetry group G* has a smaller fundamental domain, the triangle T{2, 3, 8) see figure 1, which is j^th of 
the original octagon. The method of desymmetrization can be applied to many other techniques than finite element 
methods (see [19]). 

We now focus on the four one-dimensional irreductible representations {xi)i=i---4: acting upon the generators 
as indicated in table 6. One can find in the book of Fassler and Stiefel [ , Chapter 3] the principle of desym- 
metrization in the context of dihedral symmetry. We follow their method in the case of the symmetry group G* . 
The first step is to attribute one number (value) to each of the 96 triangles that tesselate the octagon under the 
action of G* (see 2), according to the character values obtained from table 6, i.e. ±1 depending on the conjugacy 
class (remember we restrict ourselves to the one dimensional representations xi to X4)- 

Let us take the example of the first irreductible representation xi and explain how we obtain the domain and 
the boundary conditions depicted in figure 5. Table 6 shows that all 96 triangles end up with the same value, L This 
means that the eigenfunction we are looking for is even under all the 96 elements in G* and it follows that it must 
satisfy Neumann boundary conditions on all the edges of the tesselation of the hyperbolic octagon [ , ]. Finally, 
it is sufficient to solve the eigenproblem on the reduced domain T{2, 3, 8) with Neumann boundary conditions 
on its three edges. For the four one-dimensional representations one has to choose the correct combination of 
Neumann and Dirichlet boundary conditions as shown in figure 5. 

The representations of dimension > 2 require the same number of values as their dimension. For example there 
are two basis vectors determining the function values in the case of an irreductible representation of dimension 2. 
The table 6 is then no longer sufficient to set the values of the function on each triangles and one has to explicitly 
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write the matrices of the irreductible representation in order to obtain the suitable conditions. This is why we have 
restricted ourselves to the four one-dimensional representations as described in the previous paragraphs. 
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Figure 5: Boundary conditions for the one-dimensional irreducible representations. Top left: boundary conditions 
for xi, corresponding to the isotropy group G* . Top right: boundary conditions for X2 corresponding to the 
isotropy group Gqh- Bottom left: boundary conditions for xs corresponding to the isotropy group Gqk'- Bottom 
right: boundary conditions for Xi corresponding to the isotropy group G. 



5.2 Numerical experiments 

As there exists an extensive literature on the finite element methods (see for an overview [13, I ]) and as numerical 
analysis is not the main goal of this article, we do not detail the method itself but rather focus on the way to 
actually compute the eigenmodes of the Laplace-Beltrami operator 

Desymmetrized problem : For the four problems depicted in figure 5, we use the mesh generator MeshlD from 
Matlab to tesselate the triangle T(2, 3, 8) with 2995 nodes and we implement the finite element method of order 1. 
Our results are presented in figure 6 and are in a good agreement with those obtained by Balazs-Voros in [ ] and 
Aurich-Steiner in [2]. Once we have computed the eigenfunction in T(2, 3, 8), we extend it to the whole octagon 
by applying the generators of G* . We superimpose in figure 6(a) the tesselation of the octagon by the 96 triangles 
in order to allow the reader to see the symmetry class of G* . 

Non desymmetrized problem : As discussed previously, we also present some H-planforms of higher dimen- 
sion. We mesh the full octagon with 3641 nodes in such a way that the resulting mesh enjoys a I?g-symmetry, see 
figure 7. 
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(a) xi '■ G* , the corrseponding eigenvalue is A 
23.0790. 



(b) X2 : Gok, tlie corrseponding eigenvalue is A 
91.4865. 



# « 






(c) X3 • Gok'. the corrseponding eigenvalue is A 
32.6757. 




(d) X4 • G, the corresponding eigenvalue is A 
222.5434. 



Figure 6: The four H-planforms with their corresponding eigenvalue associated with the four irreductible repre- 
sentations of dimension 1, see text. 
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Figure 7: A mesh of the octagon with 3641 nodes which is used for the method of finite elements. It leads to 
matrices of dimension 3641 x 3641. 

We implement, in the finite element method of order 1, the periodic boundary conditions of the eigenprob- 
lem and obtain the first 100 eigenvalues of the octagon. Our results are in agreement with those of Aurich and 
Steiner reported in [ ]. Instead of giving a table of all the eigenvalues, we prefer to plot the staircase function 
A'^(A) = ti{'^n|An < A} for comparison with Weyl's law. Weyl's law is, in its simplest version, a statement on 
the asymptotic growth of the eigenvalues of the Laplace-Beltrami operator on bounded domains. If fi is a given 
bounded domain of M^, then the staircase function has the following asymptotic behaviour: A^(A) = ^ A -(- o(A) 
as A ^- oo. We recall that in the case of the hyperbolic octagon, one has \^\ — 4tt and hence iV(A) ^ A as A — > oo. 
As can be seen in figure 8 the asymptotic law describes the staircase well down to the smallest eigenvalues, which 
confirms the validity of our numerical results. 

We show in figure 9 two H-planforms, with D^ and ZJg^ isotropy respectively. These two H-planforms belong 
to irreductible representations of dimension 2: X5 for 9(a) and xe for 9(b) and 10(a). 

We finally present in figure 10 three H-planforms, with Csk, ^4k and D4K.' isotropy. These three H-Planforms 
belong to irreductible representations of dimension 3: xs for 10(a), X9 for 10(b) and xio for 10(c). 

In figures 6,9 and 10, we have plotted, for convenience, the corresponding H-planforms in the octagon only. 
Nevertheless, H-planforms are periodic in the Poincare disc, as stated before, and in figure 11, we plot the H- 
planform with Gqk' isotropy type of figure 6(c). We recall that the octagonal lattice group F is generated by the 
four boosts gj of subsection 4.1. Then, once the H-planform is calculated, we report it periodically in the whole 
Poincare disc by the actions of there four boosts and obtain figure 1 1 . Note that it is arduous to tesselate the entire 
disc and this is why there remains some untesselated areas in the figure. 

Remark : The finite element method relies on a variational principle and thus yields upper bounds for the 
eigenvalues. Note that it is not well-suited to compute high eigenvalues. Thus, if one wants to compute, for 
example, the first 125 eigenvalues, then the finite element method provides a very good approximation to the 
true eigenvalues. If however one wants to reach the 2000th eigenvalue by the finite element method we need to 
use matrices of size at least 10^ x 10^ which is untractable for desktop or laptop computers and requires going 
to more powerful architectures. This is why the direct boundary element method should be preferred for such 
computations (see [}]). 
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Figure 8: The staircase function N{X), in dark, is shown in comparison with Weyl's law N{X) ^ A as A — > oo. 
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(a) X5 ■ ^8: the corrseponding eigenvalue is A 
73.7323. 



(b) X8 '■ Dsk., the coiTseponding eigenvalue is A 
8.2501. 



Figure 9: Two H-planforms with their corresponding eigenvalue associated to the two irreductible representations 
of dimension 2, see text. 
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(a) xs ■ Csk, the coiTseponding eigenvalue is A 
3.8432. 



(b) X9 ■ ^4k. the corresponding eigenvalue is A 
28.0888. 
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(c) XiO ■ ^4k'i the corrseponding eigenvalue is A = 
15.0518. 

Figure 10: Three H-planforms with their corresponding eigenvalue associated to three irreductible representations 
of dimension 3, see text. 




Figure 1 1 : Extension of the Gqk.' H-planform on the Poincare disk. 
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6 Conclusion 

In this paper, we have analysed the bifurcation of periodic patterns for neural field equations describing the state of 
a system defined on the space of structure tensors, when these equations are further invariant under the isometrics 
in this space. We have made use of the concept of periodic lattice in T) (Poincare disc) to further reduce the problem 
to one on a compact Riemann surface V/T, where F is a cocompact, torsion-free Fuchsian group. Successfully, we 
have applied the machinery of equivariant bifurcation theory in the case of an octagonal periodic pattern, where 
we have been able to classify all possible H-planforms satisfying the hypotheses of the Equivariant Branching 
Lemma. In the last section, we have described a method to compute these patterns and illustrated it with a 
selection of images of octagonal H-planforms. 

There are several questions which are raised by this analysis. 

(i) The first one is that of the interpretation of these H-planforms for the modelling of the visual cortex. At this 
moment we believe that they could be involved in the process of defining texture tuning curves in a way that 
would resemble the definition of orientation tuning curves in the related ring model of orientation selectivity 
[23, 38, 18, 16, 5, 6, 39]. Another fascinating possibility is that they could be related to neural illusions 
caused by the existence of several stable stationary solutions to (1) when, e.g., the slope of the sigmoid at 
the origin becomes larger than that at which several branches of solutions bifurcate from the trivial one. 
These neural illusions would be functions of membrane potential values that would not correspond to the 
actual thalamic input and could be expressed as combinations of these H-planforms. This is of course still 
very speculative but very much worth investigating. 

(ii) The second one is about the observability of such patterns in a natural system or under direct simulation 
of the evolution equations. Indeed, not only there is a high degeneracy of the bifurcation problem if one 
removes the assumption that all perturbations respect the periodicity of the pattern, which is also the case 
for patterns in EucUdean space, but in addition the fact that such patterns would be neutral modes for the 
bifurcation problem posed in full generality in T) is non generic. We may however imagine mechanisms 
such as "spatial frequency locking" by which periodic patterns could become stable, hence observable. 

(iii) The third one is that of a more effective computation of H-planforms for a given isotropy type. As we have 
seen in the last section, a desymmetrization of the domain allowed us to calculate all the H-planforms with 
isotropy types associated to irreductible representations of dimension 1 . For irreductible representations 
of dimension > 2 the computation becomes more intricate as the desymmetrization method is no longer 
straightforward and remains misunderstood. Naturally, one solution would be to elaborate a general algo- 
rithm which, for a given isotropy type, computes systematically the associated H-planform. We think that 
such an algorithm would be of interests for quantum physisists and cosmologists. 

These questions will be the subject of further studies. 
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